{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 共享单车数据集上的线性回归分析"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 导入必要的工具包"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.metrics import mean_squared_error\n",
    "from sklearn.metrics import r2_score\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.linear_model import RidgeCV\n",
    "from sklearn.linear_model import LassoCV\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 读取数据并进行分割 (随机选择80%作为训练数据，剩下20%作为测试数据)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(584, 33) (147, 33)\n"
     ]
    }
   ],
   "source": [
    "# 从文件读取数据\n",
    "df = pd.read_csv('data/FE_day.csv')\n",
    "\n",
    "# 从做好特征工程的数据中分割出X，y\n",
    "X = df.drop(columns=['cnt','instant'])\n",
    "# X = df.drop(columns=['cnt'])\n",
    "y = df['cnt']\n",
    "\n",
    "# 进一步分割出训练集和校验集\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=23)\n",
    "print(X_train.shape, X_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 训练最小二乘线性回归"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 创建实例\n",
    "lr = LinearRegression()\n",
    "\n",
    "# 训练模型\n",
    "lr.fit(X_train, y_train)\n",
    "\n",
    "# 分别在校验集和训练集上进行预测\n",
    "y_lr_test_pred = lr.predict(X_test)\n",
    "# y_lr_train_pred = lr.predict(X_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 训练岭回归和Lasso"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 用默认alpha进行训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0\n",
      "1.5146214263573146\n"
     ]
    }
   ],
   "source": [
    "# 创建实例\n",
    "rc = RidgeCV() # RidgeCV的alphas默认设为[ 0.1  1.  10. ]\n",
    "ls = LassoCV() # LassoCV的alphas默认设为None，模型会自动设置alpha\n",
    "\n",
    "# 训练模型\n",
    "rc.fit(X_train, y_train)\n",
    "ls.fit(X_train, y_train)\n",
    "\n",
    "# 输出拟合模型后得出的最优参数\n",
    "print(rc.alpha_)\n",
    "print(ls.alpha_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用默认alphas参数，岭回归模型和Lasso模型得出的最优alpha分别是1.0和1.5，下一步则在它们附近进一步找更精确的值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 超参数alpha调优\n",
    "根据默认alphas参数得出的结果，在它们附近进一步找更精确的alpha值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RidgeCV调优后的alpha结果：0.987\n",
      "LassoCV调优后的alpha结果：1.543\n"
     ]
    }
   ],
   "source": [
    "# 将岭回归的alphas设在1.0附近\n",
    "rc_alphas = np.arange(0.5, 1.5, 0.001)\n",
    "# 将Lasso的alphas设在1.5附近\n",
    "ls_alphas = np.arange(1.0, 2.0, 0.001)\n",
    "\n",
    "# 创建实例\n",
    "rc = RidgeCV(alphas=rc_alphas, store_cv_values=True) # store_cv_values参数设为True时将保存每个alphas对应的一组误差到cv_values_属性中\n",
    "ls = LassoCV(alphas=ls_alphas)\n",
    "\n",
    "# 训练模型\n",
    "rc.fit(X_train, y_train)\n",
    "ls.fit(X_train, y_train)\n",
    "\n",
    "# 分别在校验集和训练集上进行预测\n",
    "y_rc_test_pred = rc.predict(X_test)\n",
    "# y_rc_train_pred = rc.predict(X_train)\n",
    "y_ls_test_pred = ls.predict(X_test)\n",
    "# y_ls_train_pred = ls.predict(X_train)\n",
    "\n",
    "print(f'RidgeCV调优后的alpha结果：{rc.alpha_:.3f}')\n",
    "print(f'LassoCV调优后的alpha结果：{ls.alpha_:.3f}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 可视化岭回归中不同alpha的误差"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.8870000000000005, 599917.0369487497, 'optimal alpha: 0.987')"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVIAAAEvCAYAAAAXaUnwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOy9eVxc1d34//4ACVkAWRJ2EiABEkIyEUgQQxa1cWmtca95Whutmsel2l+fr63axVZbW221Vdtqbas2WndbNe7GmEWUkAAJCWQhCCSQBZKQhSQEApzfH3NnignLALPcO5736zUv7pw5c+7nnRM+3OXcc0QphUaj0WgGT4CvA9BoNBqroxOpRqPRDBGdSDUajWaI6ESq0Wg0Q0QnUo1GoxkiOpFqNBrNEAnydQDuZsyYMSo5OdnXYZxGe3s7w4cP93UYbkG7mBN/cTGrR2lp6X6l1NiePvO7RJqcnExJSYmvwziN1tZWRo4c6esw3IJ2MSf+4mJWDxHZ0dtn+tTeSzQ0NPg6BLehXcyJv7hY0cOlRCoi4SLyuohsFZEtIpIvIpEiskxEths/I4y6IiKPi0i1iGwUkWyjfLyIlIrIBhGpFJGbu7WfIyKbjO88LiJilPe4Dyuya9cuX4fgNrSLOfEXFyt6uHpE+hjwgVJqEmADtgB3A8uVUmnAcuM9wEVAmvFaDDxplO8BzlZKTQfygLtFJN747EmjruN7Fxrlve1Do9FoTEO/iVREwoA5wNMASql2pdQhYAGwxKi2BLjU2F4APKfsrAHCRSTO+F6bUSfYsW8RiQPClFJFyv7g/3OntNXTPixHRkaGr0NwG9rFnPiLixU9XDkiTQX2Ac+KyHoR+YeIjAZilFJ7AIyf0Ub9BKC+2/cbjDJEJElENhqfP6SU2m181tBT/T72YTkCAwN9HYLb0C7mxF9crOjhyl37ICAbuF0pVSwij9H3Kbb0UKYAlFL1wDTjlP5NEXm9r/quIiKLsV8aID4+npUrVwKQmppKaGgo5eXlAERFRTFlyhRWr15tFwsKoqCggLKyMo4cOQJAbm4ujY2N1Nfb/xakpaURHBxMRUUFANHR0aSnp1NYWAhAcHAw+fn5lJSUcPToUQDy8vJoaGhwXuvJyMigoqKCYcOGARAbG0tKSgpFRUUAjBw5kry8PIqLi2ltbQUgPz+f2tpa9u7dC0BmZiadnZ1s27YNgISEBBITEykuLgYgJCSE3NxcioqKaGuzH/gXFBRQVVVFU1MTAFlZWbS1tbF9+3YAkpKSiImJcY5yCAsLIzs7m8LCQjo6OgCYM2cOlZWVHDhwAACbzcaGDRucw1OSk5OJjIykrKwMgIiICGw2G6tWrUIphYgwd+5cysvLOXjwIADZ2dk0NzdTV1fn835av349wcHBzn4KDAxk8+bNluyn0tJS591um81GS0sLNTU1luuntrY2du7c2efvky/6qU+UUn2+gFigrtv72cC7wDYgziiLA7YZ208BC7vVd9Y7pd1ngSuN727tVr4QeOrU73bfR1+vnJwcZUZWrFjh6xDchnYxJ/7iYlYPoET1knf6PbVXSu0F6kXEceHiPGAzsBRYZJQtAt4ytpcC3zXu3p8FHFZK7RGRRBEZCWDcfZ9lJMY9QIuInGXcrf/uKW31tA/LERsb6+sQ3IZ2MSf+4mJFD1cH5N8OvCAiw4Ea4Hrs11dfFZEbgJ3AVUbd94CvA9XAcaMuwGTgERFR2E/nH1ZKbTI+uwX4JzASeN94ATzYyz4sR0pKiq9DcBvaxZz4i4sVPUT52Qz5ubm5ytUnm5RSvLdpLxGjhnH2xDEejWvlypXMmzfPo/vwFtrFnPiLi1k9RKRUKZXb02df6SebRITff7iVJUV1vg5Fo9FYmK90IgWwJYVTXn/Y4/sx47PDg0W7mBN/cbGih06kieHsPXKCvYdPeHQ/eXl5Hm3fm2gXc+IvLlb0+Mon0unjwgHYUH/Io/txjE/zB7SLOfEXFyt6fOUTaWZcGMMChfIGzyZSx8Bgf0C7mBN/cbGix1c+kY4YFsjkuDA27PRsItVoNP7LVz6Rgv066aZdh+ns8txQsPz8fI+17W20iznxFxcreuhEiv3O/dG2Dr7Yd9Rj+6itrfVY295Gu5gTf3GxoodOpMD0JM/fcHJMluAPaBdz4i8uVvTQiRRIHTOa0BFBlHv4zr1Go/FPdCIFAgIEW2K4R49IMzMzPda2t9Eu5sRfXKzooROpgS3pDLbubeHEyU6PtN/Z6Zl2fYF2MSf+4mJFD51IDWyJ4XR2KSp3e+ZxUccEsv6AdjEn/uJiRQ+dSA0cN5zW6/GkGo1mgOhEahAdNoL4M0ZQ3uCZI9KEhIT+K1kE7WJO/MXFih46kXZj+rhwNtQf9EjbiYmJHmnXF2gXc+IvLlb00Im0G7bEcOqbWzlw1IXFrgaIFSdi6A3tYk78xcWKHjqRdsNmXCfd6KHTe41G45/oRNqNqQlnECCw3gPjSUNCQtzepq/QLubEX1ys6PGVXrOpJy58dDXRYSN47nsz3RiVRqOxOnrNpgGQPT6C9TsP0uXmmaCKiorc2p4v0S7mxF9crOihE+kpZI+LoOVEB9Vungmqrc39N7B8hXYxJ/7iYkUPnUhPIWd8BAClOzwzDEqj0fgfOpGeQnLUKCJHD6fMzYm0oKDAre35Eu1iTvzFxYoeOpGegoiQPS6c0p3uTaRVVVVubc+XaBdz4i8uVvTQibQHssdHULPvGM3H2t3WZlNTk9va8jXaxZz4i4sVPXQi7YGccfbrpOvdfFSq0Wj8E51Ie2BaYjhBAUKZGxNpVlaW29ryNdrFnPiLixU9XEqkIhIuIq+LyFYR2SIi+SISKSLLRGS78TPCqCsi8riIVIvIRhHJNsqni0iRiFQa5d/q1n6KiBQbbb0iIsON8mDjfbXxebL7/wlOZ+TwQDLjw9x6596KQzp6Q7uYE39xsaKHq0ekjwEfKKUmATZgC3A3sFwplQYsN94DXASkGa/FwJNG+XHgu0qpKcCFwKMiEm589hDwR6Otg8ANRvkNwEGl1ETgj0Y9r5A9LoLy+sN0dHa5pb3t27e7pR0zoF3Mib+4WNGj30QqImHAHOBpAKVUu1LqELAAWGJUWwJcamwvAJ5TdtYA4SISp5SqUkptN9rYDTQBY0VEgHOB13tpy7GP14HzjPoeJ3t8BK0nO9m6t8Ubu9NoNBbGlSPSVGAf8KyIrBeRf4jIaCBGKbUHwPgZbdRPAOq7fb/BKHMiIjOB4cAXQBRwSCnV0UN9Z1vG54eN+h7H3QPzk5KS3NKOGdAu5sRfXKzoEeRinWzgdqVUsYg8xn9P43uipyNG54PrIhIHPA8sUkp19XKE6ajfZ1vd2lyM/TIC8fHxrFy5EoDU1FRCQ0MpLy8HICoqiilTprB69Wq7WFAQBQUFlJWVceTIEQByc3NpbGxk586dRAQLhVt38Y30ECoqKgCIjo4mPT2dwsJCAIKDg8nPz6ekpISjR+2Plebl5dHQ0MCuXbsAyMjIICgoyBlXbGwsKSkpzmeKR44cSV5eHsXFxbS2tgKQn59PbW2tc43vzMxMOjs7nevZJCQkkJiY6Jy7MSQkhNzcXIqKipzXmAoKCqiqqnIOJ8nKyqKtrc156pSUlERMTAyOSV7CwsLIzs6msLCQjg7737U5c+ZQWVnJgQMHALDZbHR1dTldkpOTiYyMpKysDICIiAhsNhurVq1CKYWIMHfuXMrLyzl40P5HKTs7m+bmZurq6obcT/X19r/ZaWlpBAcHD7ifTpw44XTJyMggMDCQzZs3W7KfDh8+7HSx2Wy0tLRQU1NjuX6aOnVqv79PvuinPlFK9fkCYoG6bu9nA+8C24A4oywO2GZsPwUs7Fa/e70woAy4qtvnAuwHgoz3+cCHxvaHQL6xHWTUk77izcnJUe7iln+VqFkPLndLWytWrHBLO2ZAu5gTf3ExqwdQonrJO/2e2iul9gL1IpJhFJ0HbAaWAouMskXAW8b2UuC7xt37s4DDSqk9xp34N7BfP32tW/sKWAFc2Utbjn1cCXxi1PcK2eMiaDjYSuORE97apUajsSCunNoD3A68YCTDGuB67NdXXxWRG4CdwFVG3feArwPV2O/UX2+UX439plWUiFxnlF2nlNoA3AW8LCK/BtZj3Ngyfj4vItVAM3DNYCQHi+M6admOg1w0NW5IbYWFhbkjJFOgXcyJv7hY0UNP7NwH7R1dZP3yQxblj+en38h0S5sajcaa6ImdB8nwoACmJZxBiRvu3DsupvsD2sWc+IuLFT10Iu2H3ORINjUcprW9c0jtOO6u+gPaxZz4i4sVPXQi7YeZKRF0dCnWe2i9e41GY310Iu2HnPGRiMC62qEl0jlz5rgpIt+jXcyJv7hY0UMn0n44Y+QwJsWGsa6ueUjtVFZWuiki36NdzIm/uFjRQydSF5iZHEHZzoOcHMIEJo4ng/wB7WJO/MXFih46kbrAjJRIjrd3Urn7iK9D0Wg0JkQnUheYmRwJwLrawZ/e22w2d4Xjc7SLOfEXFyt66ETqAtFhIxgfNYq1Q7hO2tLiP9PxaRdz4i8uVvTQidRFZiRHUlLXTFfX4J4Ec8zC4w9oF3PiLy5W9NCJ1EVmJkdy8PhJvth31NehaDQak6ETqYvMSLFfJx3s6X1ycrIbo/Et2sWc+IuLFT10InWR5KhRjAkJHvQNp8jISDdH5Du0iznxFxcreuhE6iIiQl5KJOvqBveEk2Nmcn9Au5gTf3GxoodOpANgRnIEuw610nDwuK9D0Wg0JkIn0gHguE46mMdFIyIi3B2Oz9Au5sRfXKzooRPpAJgUG0ZocBBrBzGBiRUHGfeGdjEn/uJiRQ+dSAdAYICQkxzB2tqBPwu8atUqD0TkG7SLOfEXFyt66EQ6QM5KjeKLfcdoahnYgnj+tKSLdjEn/uJiRQ+dSAdIfmoUAGtqBnadVEQ8EY5P0C7mxF9crOihF78bIB2dXZx5/zIutsXz28unemw/Go3GXOjF79xIUGAAM1MiWVMzsOuk5eXlHorI+2gXc+IvLlb00Il0EJyVGkXt/mPsPez6ddKDB/1nzSftYk78xcWKHjqRDoL8CY7rpNabyVuj0bgfnUgHweS4MMJGBFH0heuJNDs724MReRftYk78xcWKHjqRDoLAACEvNYqiARyRNjcPbfE8M6FdzIm/uFjRQyfSQZKfGsXO5uPsOtTqUv26ujrPBuRFtIs58RcXK3roRDpInNdJB3B6r9Fo/BOXEqmIhIvI6yKyVUS2iEi+iESKyDIR2W78jDDqiog8LiLVIrJRRLK7tfOBiBwSkXdOaT9FRIqNtl4RkeFGebDxvtr4PNl96kMjIyaUiFHDXD69T01N9XBE3kO7mBN/cbGih6tHpI8BHyilJgE2YAtwN7BcKZUGLDfeA1wEpBmvxcCT3dr5PXBtD+0/BPzRaOsgcINRfgNwUCk1EfijUc8UBAQIeSlRLt9wCg0N9XBE3kO7mBN/cbGiR7+JVETCgDnA0wBKqXal1CFgAbDEqLYEuNTYXgA8p+ysAcJFJM747nKg5ZT2BTgXeL2Xthz7eB04T0z0/Fj+hCh2HWqlvrn/+UmtOMi4N7SLOfEXFyt6uHJEmgrsA54VkfUi8g8RGQ3EKKX2ABg/o436CUB9t+83GGW9EQUcUkp19FDf2Zbx+WGjvilwXCcdyDAojUbjfwS5WCcbuF0pVSwij/Hf0/ie6OmIsa8H+vuq71JbIrIY+2UE4uPjWblyJWC/1hIaGur8CxcVFcWUKVNYvXo1AEFBQRQUFFBWVsaRI0cAyM3NpbGxkfp6+9+CtLQ0goODqaioACA6Opr09HQKCwtRShEWLKypOUCqNHL0qH2F0by8PBoaGti1axcAGRkZjBw50hlXbGwsKSkpFBUVATBy5Ejy8vIoLi6mtdU+CiA/P5/a2lr27t0LQGZmJp2dnWzbtg2AhIQEEhMTKS4uBiAkJITc3FyKiopoa2sDoKCggKqqKpqamgDIysqira2N7du3A5CUlERMTAyOuQnCwsLIzs6msLCQjg7737U5c+ZQWVnJgQP2PxY2m43hw4c7XZKTk4mMjHQuDxEREYHNZmPVqlUopRAR5s6dS3l5ufOJlezsbJqbm513Zz3dTwDBwcHk5+dTUlLypX4KDAx0umRkZBAYGMjmzZst2U9KKaeLzWajpaXFubSxlfopKirqtH469ffJF/3UF/1OWiIiscAapVSy8X429kQ6EZinlNpjnLqvVEpliMhTxvZLRv1tjnrG+3nAnUqpi433gv2IN1Yp1SEi+cAvlVIXiMiHxnaRiAQBe4Gxqo+gPT1pyanc9mIZpXUHKbrn3D5nrenq6iIgwD8GSWgXc+IvLmb1GNKkJUqpvUC9iGQYRecBm4GlwCKjbBHwlrG9FPiucff+LOCwI4n20r4CVgBX9tKWYx9XAp/0lUR9QcHEMew9cqLf9e4df7X9Ae1iTvzFxYoerpzaA9wOvGAMS6oBrseehF8VkRuAncBVRt33gK8D1cBxoy4AIvIpMAkIEZEG4Aal1IfAXcDLIvJrYD3GjS3j5/MiUg00A9cMVtRTFEwcA0Dh9v1MjLbe3UaNRjN0XEqkSqkNQE+HtOf1UFcBt/XSzuxeymuAmT2Un+C/CdqUJEWOYnzUKAqr93PdrJRe6wUFufo3y/xoF3PiLy5W9NATO7uBn76xibc27Gb9vfMZFmi+azsajWbo6ImdPczstDEcbetgQ/2hXus47pb6A9rFnPiLixU9dCJ1A/mpYwgQ+3XS3nAMB/EHtIs58RcXK3roROoGzhg1jKmJ4RRW955INRqN/6ITqZuYPXEMG+oPceTEyR4/z83t8dKKJdEu5sRfXKzooROpmyhIG0Nnl+p1Wr3GxkYvR+Q5tIs58RcXK3roROomzhwXzshhgb2e3jsekfMHtIs58RcXK3roROomgoMCyUuN1NdJNZqvIDqRupGCiWOo2XeM3T0sP5KWluaDiDyDdjEn/uLiDY/W9k63tqcTqRuZnTYW6HkYVHBwsLfD8RjaxZz4i4unPaoaW5jxwMesrtrntjZ1InUj6TEhjA0N5tMeTu8d04b5A9rFnPiLi6c9nims5WRnF1kJZ7itTZ1I3YiIUDBxDJ9V76ery78evdVo/IEDR9v4z/pdXJ6dSOTo4W5rVydSNzMnfQzNx9rZtOvwl8qjo6N7+Yb10C7mxF9cPOnxQvFO2ju6uKEg2a3t6kTqZuakjUUEVm778vWX9PR0H0XkfrSLOfEXF095tHV08lzRDuZljHX7lJc6kbqZqJBgpiWGs7Kq6UvljqUU/AHtYk78xcVTHm+X72H/0TZuKOh9usvBohOpB5iXPpYN9Yc4eKzd16FoNBpAKcXThbVkxIQ6J2N3JzqReoB5GWNRClZv/+/pvb8MTQHtYlb8xcUTHkU1B9iy5wjfK0juc221waITqQeYlhhOxKhhrOp2nTQ/P9+HEbkX7WJO/MXFEx7PFNYSNXo4C6b3tTL84NGJ1AMEBghz0seyqmqfcxiUt2ft9yTaxZz4i4u7PWr2HWX51ia+fdZ4RgwLdGvbDnQi9RDzMsZy4Fg7Fbvtw6Aca3T7A9rFnPiLi7s9nv2sjmEBAVx71ni3ttsdnUg9hGMY1Iqt7nsMTaPRDIxDx9t5vbSBS6bHMzbUc9eQdSL1EFEhwUxLOMM5DCovL8/HEbkP7WJO/MXFnR4vra2n9WQn3+tjhV93oBOpB5mbEe0cBtXQ0ODrcNyGdjEn/uLiLo/2ji6WfF7H2ROiyIwPc0ubvaETqQfpPgxq165dvg7HbWgXc+IvLu7yeLt8N3uPnGDxnFS3tNcXOpF6EFsPw6A0Go3nUUrxt9U1TIoNZW76WI/vTydSDxIYIMxOsw+DSvOT56ABMjIyfB2C29Au5sMdHiur9rGtsYWbZqd6ZAD+qehE6mHOnRTNgWPtbG064etQ3EZgoGfG4vkC7WI+3OHxt1U1xJ0xgm/a4t0QUf/oROph5mWMJTBAeGNtta9DcRubN2/2dQhuQ7uYj6F6bGw4RFHNAb43K4XhQd5JcS7tRUTCReR1EdkqIltEJF9EIkVkmYhsN35GGHVFRB4XkWoR2Sgi2d3aWWTU3y4ii7qV54jIJuM7j4txLN7bPqxE+KjhzEiOYENTh69D0Wi+Ejy1uobQ4CCumZnktX26mq4fAz5QSk0CbMAW4G5guVIqDVhuvAe4CEgzXouBJ8GeFIFfAHnATOAX3RLjk0Zdx/cuNMp724el+NrkGBqOKuqbj/s6FLcQGxvr6xDchnYxH0Px2HngOO9v2sP/nDWO0BHD3BhV3/SbSEUkDJgDPA2glGpXSh0CFgBLjGpLgEuN7QXAc8rOGiBcROKAC4BlSqlmpdRBYBlwofFZmFKqSCmlgOdOaaunfViK8ybHAPDxlkYfR+IeUlI8O7jZm2gX8zEUj6cLawgMEI8PwD8VV45IU4F9wLMisl5E/iEio4EYpdQeAOOnY32ABKC+2/cbjLK+yht6KKePfViKlDGjiR8tfpNIi4qKfB2C29Au5mOwHs3H2nmlpJ5LpycQEzbCzVH1TZCLdbKB25VSxSLyGH2fYvc01kANotxlRGQx9ksDxMfHs3LlSgBSU1MJDQ2lvLwcgKioKKZMmcLq1asBCAoKoqCggLKyMo4cOQJAbm4ujY2N1Nfbc35aWhrBwcHOlQ2jo6NJT093zuIdHBxMfn4+JSUlzskW8vLyaGhocA4szsjIYNoYYdkXB3hv2QpSk+JISUlx/ocZOXIkeXl5FBcX09raCtinEqutrWXv3r0AZGZm0tnZybZt2wBISEggMTGR4uJiAEJCQsjNzaWoqIi2tjYACgoKqKqqoqnJ/phqVlYWbW1tbN++HYCkpCRiYmKcs+2EhYWRnZ1NYWEhHR32a7pz5syhsrKSAwcOAGCz2Whvb3f+GycnJxMZGUlZWRkAERER2Gw2Vq1ahVIKEWHu3LmUl5dz8OBBALKzs2lubqaurs7n/dTW1uZ0ycjIIDAw0HmzIzY21lL91Nra6nSx2Wy0tLRQU1NjuX4C+v196qmffvXSKk6c7OKscPv33N1PfaKU6vMFxAJ13d7PBt4FtgFxRlkcsM3YfgpY2K3+NuPzhcBT3cqfMsrigK3dyp31ettHX6+cnBxlRv757qdq/F3vqKUbdvk6lCGzZs0aX4fgNrSL+RiMR2t7hzrz/o/U9c+u9UBEdoAS1Uve6ffUXim1F6gXEcco2fOAzcBSwHHnfRHwlrG9FPiucff+LOCwsp+WfwicLyIRxk2m84EPjc9aROQs4279d09pq6d9WI7vXDiLyNHD/eL03l8mxwDtYkYG4/FaaQPNx9r5Xy88DtoTrt61vx14QUQ2AtOB3wAPAvNFZDsw33gP8B5QA1QDfwduBVBKNQO/AtYZr/uNMoBbgH8Y3/kCeN8o720flqNk3VrOnRTNiq1NnOzs8nU4Q8Jx+uMPaBfzMVCPzi7FPz6twZYUzsyUSA9F1TeuXCNFKbUByO3ho/N6qKuA23pp5xngmR7KS4CsHsoP9LQPK9La2srXJo/j9dIGSuoOkj8hytchDRrHdSd/QLuYj4F6vLtpDzsOHOeeiyZ55XHQntBPNnmR2WljGR4Y4Ben9xqNGVBK8cSKaiaMHc35mb4bR6sTqZfIz89ndHAQZ0+M4uMtjY4ba5bEXxZZA+1iRgbi8cnWJrbubeHWeRMJCPDN0SjoROo1amtrAftTTjsOHKeq0brr6zhc/AHtYj5c9VBK8ecV1SSEj+SS6d6ZnKQ3dCL1Eo7xa+dnxiACH1Ts9XFEg8fh4g9oF/PhqseammbW7zzEzXNTGRbo21SmE6mXiQ4bQc64CD6o9I//9BqNr3hiZTVjQoK5Ktd7k5P0hk6kXiIzM9O5fWFWLFv2HGHHgWM+jGjwdHexOtrFfLjiUV5/iE+37+fG2SkeW6t+IOhE6iU6Ozud2xdMsd9dtOrpfXcXq6NdzIcrHk+srCZsRBDfzhvnhYj6RydSL+F4phcgKXIUWQlhlj297+5idbSL+ejPY3tjCx9WNnLd2clenSqvL3Qi9REXZcWxfuch9hz2j0HUGo23eHLlF4wcFsh1Xp4qry90IvUSCQkJX3rvOL3/qNJ6g/NPdbEy2sV89OVR33yct8p38z9544gcPdyLUfWNTqReIjEx8UvvJ0aHkBYdYsnrpKe6WBntYj768nhq9RcECNw02zeTk/SGTqReoqeJGC7MiqW49gAHjrow36GJ8JfJMUC7mJHePBqPnODVkgauyE4k9gzvTtzcHzqR+pALs2LpUv6zBIlG40meXPkFnV2KW+dN9HUop6ETqZcICQk5rSwzLoykyJG8b7HT+55crIp2MR89eTQdOcFLa3dy+ZkJjIsa5YOo+kYnUi+Rm3v6LIQiwtez4visej+Hjrf7IKrB0ZOLVdEu5qMnjydXfUFHl+L755rvaBR0IvUavS3odfG0eE52Kj600JhSf1lkDbSLGTnVo+nICV4s3sllZyYwPmq0j6LqG51IvURvC2hlJYQxPmoU72zc4+WIBo9Li4FZBO1iPk71eGp1jf1o9BxzHo2CTqQ+R0T45rR4Pqvez36L3b3XaDzNvpY2XijewYLp8SSPMefRKOhE6jUKCgp6/exiWxxdCsvcdOrLxds8+uijHD9+3Pn+61//OocOHXL5+725rFy5kosvvtjldurq6sjKOm21nAHXGQhLliwhLS2NtLQ0lixZ0qNLeXk5+fn5TJ06lW9+85vOZZJPnjzJokWLmDp1KpMnT+a3v/0tYH88c/r06c5XWFgYjz76qNtidoXuHn9b/QXtHV3cfm6aV2MYKDqReomqqqpeP8uICSUtOoS3y3d7MaLB05eLtzk1kb733nuEh4e7/H0zuQyE5uZm7rvvPoqLi1m7di333Xcf69atO63ejTfeyIMPPsimTZu47LLL+P3vfw/Aa6+9RltbG5s2baK0tJSnnnqKuro6MjIy2LBhAxs2bKC0tJRRo0Zx2WWXedXN0Sf7j7bx/JodXDo9gRQTH42CTqReo6mpqdfPRISLp8Wzrq6ZvYdPeDGqwdGXy0WiZ38AACAASURBVFD5wx/+QFZWFllZWc4jobq6OiZNmsSiRYuYNm0aV155JcePH+fxxx9n9+7dnHPOOZxzzjkAJCcns3//fud3brzxRrKysvj2t7/Nxx9/zKxZs0hLS2Pt2rUArFq1irPPPpszzzyTs88+u98JM+rq6pg9ezbZ2dlkZ2fz+eefn1bnn//8JwsWLODCCy8kIyOD++67z/lZZ2cnN910E1OmTOH88893LvT297//nRkzZmCz2bjiiiu+9MehJz788EPmz59PZGQkERERzJ8/n/fee++0etu2bWPOnDkAzJ8/n3//+9+A/f/csWPH6OjooLW1leHDhxMWFval7y5fvpwJEyYwfvz4PmNxN47/X39bXUN7R5dp79R3RydSk3CxLQ6l7CsiflUpLS3l2Wefpbi4mDVr1vD3v/+d9evXA/aEsHjxYjZu3EhYWBhPPPEEd9xxB/Hx8axYsYIVK1ac1l51dTU/+MEP2LhxI1u3buXFF1+ksLCQhx9+mN/85jcAjBs3jtWrV7N+/Xruv/9+fvKTn/QZY3R0NMuWLaOsrIxXXnmFO+64o8d6a9eu5YUXXmDDhg289tprlJSUALB9+3Zuu+02KisrCQ8Pdya2yy+/nHXr1lFeXs7kyZN5+umnAVi6dCn33nvvae3v2rWLpKT/TmicmJjI/v37T6uXlZXF0qVLAftRaH19PQBXXnklo0ePJi4ujnHjxnHnnXcSGfnlpYxffvllFi5c2Oe/h6fYf7SN54t2cIktntSx5h8fqxOpl+jv2tiEsSFkxoXxzkbzn9678zpfdwoLC7nssssYPXo0ISEhXH755Xz66acAJCUlMWvWLAC+853vUFhY2G97KSkpTJ06lYCAAKZMmcJ5552HiDB16lTq6uoAewK66qqryMrK4oc//CGVlZV9tnny5Eluuukmpk6dylVXXcXmzZt7rDd//nyioqIYOXIkl19+uTPelJQUpk+fDkBOTo4zjoqKCmbPns3UqVN54YUXnHFccskl3H///ae139PiiXFxcaeVPfPMM/zlL38hJyeHlpYWhg+3T/Sxdu1aAgMD2b17N7W1tTzyyCPU1NQ4v9fe3s7SpUu56qqr+vz38ARZWVn8/dMaTnR08n2TXxt1oBOpl3BlaMo3bfGs33mI+ua+T+t8jaeG2fS1suqp65W7sn55cHCwczsgIMD5PiAggI6ODgAeeOABzjnnHCoqKnj77bc5caLvSyt//OMfiYmJoby8nJKSEtrbe36Qord4u8cUGBjojOO6667jz3/+M5s2beIXv/hFv3EkJiY6jy4BGhoaiIqKOq3epEmT+OijjygtLWXhwoVMmDABgBdffJELL7yQYcOGER0dzaxZs5xHzQDvv/8+2dnZxMTE9BmHJ9hz8CjPfW4/Gp0Ybf6jUdCJ1Gts37693zoXT7MfUZh9TKkrLoNhzpw5vPnmmxw/fpxjx47xxhtvMHv2bAB27tzpHKj90ksvOe/shoaG0tLSMuh97tmzxzlt2z//+c9+6x8+fJi4uDgCAgJ4/vnne53NfdmyZTQ3N9Pa2sqbb77pPJrujZaWFuLi4jh58iQvvPBCv3FccMEFfPTRRxw8eJCDBw/y0UcffelU34HjemNXVxe//vWvufnmmwH7JY1PPvkEpRTHjh1jzZo1TJo0yfm9l156yWen9Y9/vI32zi5+cJ41jkZBJ1JTkRQ5ijPHhfPWhl2+DsUnZGdnc9111zFz5kzy8vK48cYbOfPMMwGYPHkyS5YsYdq0aTQ3N3PLLbcAsHjxYi666CLnzaaBcs0113DPPfcwa9Ysl5a4uPXWW1myZAlnnXUWVVVVjB7d893kgoICrr32WqZPn84VV1zR7+Obv/rVr8jLy2P+/PlfSmi9XSONjIzk5z//OTNmzGDGjBnce++9zptFN954o/Po8qWXXiI9PZ1JkyYRHx/P9ddfD8Btt93G0aNHycrKYsaMGVx//fVMmzYNgOPHj7Ns2TIuv/zyfv893M3uQ62s2NnBFdkJlrg26kQp5VevnJwcZUaqq6tdqrfk81o1/q531Obdhz0c0eBx1cVd1NbWqilTpnikbU+4PPvss+q2225ze7v94e1+8QR3/3ujmnDPO6q++ZivQzkNoET1knf0EamXcPVa08XT4gkKEN5Yb96jUl9cN/MU2sU87DhwjNdK6rk6O57ECPPN8NQXLiVSEakTkU0iskFESowym4gUGeVvi0iYUT5cRJ41ystFZF63dr4lIhtFpFJEftetPFhEXhGRahEpFpHkbp/dY5RvE5EL3OTtdbpfyO+LyNHDmZcRzVsbdtHZ1fvNF1/iqstA6e3GTXJyMhUVFR7ZpydcHDeOvI2n+sVbPPrxdoIChZkhB30dyoAZyBHpOUqp6Uopx8WefwB3K6WmAm8APzLKbwIwyucDj4hIgIhEAb8HzlNKTQFiROQ84zs3AAeVUhOBPwIPAYhIJnANMAW4EHhCRHy/iLWHuTw7gcYjbXz+xenjAv2Z4OBgVq5c6eswND5ge2MLb27YxaL8ZMJHWO9EeSgRZwCrje1lwBXGdiawHEAp1QQcAnKBVKBKKbXPqPdxt+8sAJYY268D54l9vMgC4GWlVJtSqhaoBmYOIWafcepTI31x7qRoQkcE8UaZOU/vB+IyUH796197rO2e8KSLt7Gyyx+WVTF6eBD/O3eCJT1cTaQK+EhESkVksVFWAVxibF8FOMZelAMLRCRIRFKAHOOzamCSiCSLSBBwabfvJAD1AEqpDuAwENW93KDBKLMc2dnZLtcdMSyQb0yN44PKvRxv7/BgVINjIC4DIS0tjeXLl3uk7d7wlIsvsKpLxa7DvF+xl+8VpBA5erglPYJcrDdLKbVbRKKBZSKyFfge8LiI3AssBRwXuJ4BJgMlwA7gc6BDKXVQRG4BXgG6jHLHUoA9ja5WfZR/CSO5LwaIj493nh6mpqYSGhpKeXk5AFFRUUyZMoXVq+0H0kFBQRQUFFBWVuacFSc3N5fGxkbnYOe0tDSCg4Od1+iio6NJT093PqkSHBxMfn4+JSUlHD16FIC8vDwaGhrYtct+RJmRkcGWLVsICLD/3YqNjSUlJcU5LnLkyJHk5eVRXFzsfPb64qxJvLyunkdfX8nZ8UFkZmbS2dnpfBY8ISGBxMRE50JhISEh5ObmUlRU5BwwX1BQQFVVlXMsYVZWFm1tbc5xoElJScTExDivrYWFhZGdnU1hYaFzoPicOXOorKzkwIEDANhsNkpLSwkKsv/XSU5OJjIykrKyMgAiIiKw2WysWrUKpRQiwty5cykvL+fgQfu1r+zsbJqbm51P9Tj6acGCBTz88MNs2rTJa/1UWFjIsGHDnP0UGBjofFrJlX7Kz8+ntraWvXvtM3f5sp+WLVvmHPBvs9loaWlxPq3kzn5y9+/T79e0MHoYTGIXRUUH6OzsZMSIEX3+Pvmin/qkt9v5vb2AXwJ3nlKWDqztpf7nQGYP5YuB3xnbHwL5xnYQsB97Er0HuKfbd5z1enuZdfjTihUrBlS/s7NLnf3b5eo7/1jjmYCGwEBdXOXEiRMKUFVVVR5pvyc85eILrOiyrvaAGn/XO+ovK7Y7y8zqwVCGP4nIaBEJdWwD5wMVxtEpIhIA/Az4q/F+lFEPEZmP/Wh0s/He8Z0I4FbsN6zAfkS7yNi+EvjECHwpcI1xVz8FSAPW9v/nwfoEBAiXnZnAZ9X7aTpi/hmh3IHjaOrhhx/2cSQab6CU4sH3tzImJJjrzk72dThDo7cMq/57FJiK/bpnOVAJ/NQo/wFQZbweBMQoTwa2AVuw31Aa362tl4DNxuuabuUjgNewX0ddC6R2++ynwBdGmxf1F69Zj0g7OzsH/J3qphY1/q531F9Xmmug9WBcXCUgIEDZ/1t6B0+6eBuruXxUuVeNv+sd9XxR3ZfKzerBUI5IlVI1Simb8ZqilHrAKH9MKZVuvO42doRSqk4plaGUmqyU+ppSake3thYqpTKN18vdyk8opa5SSk1USs1UStV0++wBpdQEo833Xf4LYTL6m1WoJyaMDeHMceG8Vtrg+KNiCgbj4ip33XWXx9ruCU+6eBsruXR0dvHQB1tJHTOab8348hwBVvJwYL0BWxbFcbNmoHwrN4nqpqOU7XR9+QxPM1gXV3DM73n48GGP7aM7nnTxNlZyeb20geqmo/z4wgyGBX45DVnJw4FOpCbnYls8o4YH8uq6+v4r+wGxsbEAPPXUUz6OROMpWts7+ePHVZw5LpwLpsT6Ohy3oBOpl7DZbIP6XkhwEN+YGsc7G3dzrM0cY0oH6zIQHnroIY/vA7zj4i2s4vLMZ7U0Hmnjnosm9zivrFU8uqMTqZcYypyZ35qRxLH2Tt41yTylQ3Fxhauvvprm5maP7sOBp128iRVcmo+189eVX/C1ydHMTInssY4VPE5FJ1Iv0X0Zh4GSMz6C1LGjeaXEHKf3Q3FxhR//+McALs0POlQ87eJNrODylxXVHGvv4McXTuq1jhU8TkUnUgsgInwrN4nSHQepbrLeX+uBkpOTA8Cbb77p40g07qS++TjPF+3gypxE0mNCfR2OW9GJ1EskJycP6fuXZycSFCC8WtLgnoCGwFBdXMUb10m95eINzO7yyEfbEIEfzk/vs57ZPXpCJ1IvcepStwNlbGgw506K5j9lDZzs7HJTVINjqC6ukJOTw7p16zy+H2+4eAszu2xqOMybG3bzvYIU4s4Y2WddM3v0hk6kXsIxWcRQuGZmEvuPtvPx5kY3RDR43OHSH94amO8NF29hVhelFPe/U8mYkOHcOm9Cv/XN6tEXOpFaiLnp0SSEj+SF4p2+DsXjOBZeW79+vY8j0QyV9yv2sq7uIP83P4PQEcN8HY5H0InUS0RERAy5jcAAYeHMJAqr91Oz76gbohoc7nDpj8BA+0IIv/vd7/qpOTS84eItzOhy4mQnv31/C5NiQ097FLQ3zOjRHzqRegl3DTK+ekYSwwLFp0el3howfcYZZ/Dyyy/3X3EIWHHwd2+Y0eWfn9dR39zKz76RSWBAT9MLn44ZPfpDJ1IvsWrVKre0Ex06ggumxPJaST2t7Z4fZ9kT7nLpj7vvvtvj+/CWizcwm8u+ljb+/Ek1X5scTUHaGJe/ZzYPV9CJ1Eu4c/ama88az5ETHbxdvtttbQ4Eb81EdfPNNwPQ2Oi5m2tmmlVrqJjN5Q/LqjhxspOffH3ygL5nNg9X0InUS/T0TPFgmZkSSXpMCP8q3tF/ZQ/gTpe+CA8PB+BPf/qTx/bhLRdvYCaXrXuP8Mq6nVybP57UsSED+q6ZPFxFrJj9+yI3N1dZfX1vV3iuqI5736rkrdtmYUsK93U4HkNECAoK4uTJk74OReMiSimufXotm3YdZtWP5hE+arivQ3ILIlKq/rsc/ZfQR6RewrFgmLu47MwERg0P5F9rvH9U6m6XvrjhhhucC7x5Am+6eBqzuCzf0kRh9X5++LW0QSVRs3gMBJ1IvYRjZUZ3ETpiGAumJ7C0fDcHj7X3/wU34m6XvvjRj34E4NpKjoPAmy6exgwuJ052ct87laRFh/Dts8YPqg0zeAwUnUgtzKKzx9PW0cVL6/x3gH5GRgYAL774oo8j0bjC31bXUN/cyn2XTDlt5nt/5qtj6mOys7Pd3uak2DBmTYziuc93ePX5e0+49IenJjDxhYun8LVLffNx/rKimm9Mi+Psia4PdzoVX3sMBp1IvYSnJiq+oSCFvUdO8N4m70367K1Jlx3MmzePbdu2eaRtb7t4El+7/PrdzQSI8NMBDnc6FV97DAadSL1EXV2dR9qdlx5N6pjRPF1Y67Xxd55y6Q3HBCae8PO2iyfxpcuqqn18WNnI98+dSHx437M79YcV+0QnUosTECBcPyuZjQ2HKd1hvYv0rnDBBRcA8Omnn/o4Ek1PtHd0cd/SSlLGjObG2Sm+Dscn6ETqJVJTUz3W9hU5iZwxchhPF9Z6bB/d8aRLTzgGaHviOqm3XTyJr1yeLqylZv8xfvHNTIKDAofcnhX7RCdSLxEa6rmlFUYND2LhzHF8WLmX+ubjHtuPA0+69EZSUhLvvfee29v1hYun8IXLnsOt/OmT7czPjGFeRrRb2rRin+hE6iU8Pcj4u/njERGWfF7n0f2AbwZMe2qiZysO/u4NX7j86p3NdHYp7r04021tWrFPdCL1E+LDR/L1qXG8vK6ew63+9zjl9ddfD1jzRoS/snxLI+9t2ssd56WRFDnK1+H4FJ1IvURUVJTH9/G/c1I52tbh8cdGveFyKqNG2X9R//CHP7i1XV+4eApvuhxv7+Det+xPMN00273XNK3YJy4lUhGpE5FNIrJBREqMMpuIFBnlb4tImFE+XESeNcrLRWRet3YWGuUbReQDERljlEeKyDIR2W78jDDKRUQeF5Fq4zvWG6lrMGXKFI/vIyvhDOakj+XZz2o5cdJzc5V6w6U33D0TlC9d3I03XR79eDu7DrXym8unMjzIvcdjVuyTgfwLnKOUmt5t9pN/AHcrpaYCbwA/MspvAjDK5wOPiEiAiAQBjxntTAM2At83vnM3sFwplQYsN94DXASkGa/FwJODcDQFq1ev9sp+bpk7gf1H23mtpN5j+/CWy6n88Ic/dHubvnLxBN5yqdx9mKcLa1k4M4kZye5f8dOKfTKUPyUZgMN4GXCFsZ2JPRmilGoCDgG5gBiv0WIfzxIGOGYmXgAsMbaXAJd2K39O2VkDhItI3BBi9nvOSo3kzHHhPLW6hg4fL9vsbhyJ9OhR361X9VWns0vxkzcqiBg1jLsunOTrcEyDq4lUAR+JSKmILDbKKoBLjO2rAMfKVuXAAhEJEpEUIAdIUkqdBG4BNmFPoJnA08Z3YpRSewCMn45xFAlA90OrBqPMcgQFBXllPyLCrfMm0nCwlXc2euaxUW+5nEpSkv2/2NNPP91PTdfxlYsn8IbLC8U7KK8/xM8vzvTYPKNW7BNXI56llNotItHAMhHZCnwPeFxE7gWWAo653J4BJgMlwA7gc6BDRIZhT6RnAjXAn4B7gF/3sd+epso+7TlBI7kvBoiPj2flypWAfWBvaGioczhFVFQUU6ZMcZ46BAUFUVBQQFlZGUeOHAEgNzeXxsZG6uvt+TstLY3g4GAqKioAiI6OJj09ncLCQgCCg4PJz8+npKTEeaSUl5dHQ0MDu3btAuwzGKWnpzvjio2NJSUlhaKiIgBGjhxJXl4excXFtLa2ApCfn09tbS179+4FIDMzk87OTucz5wkJCSQmJlJcXAxASEgIubm5FBUVEXjiBPEhwhMrq0kbfoh9+/YBkJWVRVtbG9u3bwfsiSkmJgbHRNhhYWFkZ2dTWFjonAN0zpw5VFZWcuDAAcC+MNm4ceOcLsnJyURGRjrXIo+IiMBms7Fq1SqUUogIc+fOpby83Dk9WnZ2Ns3Nzc478APpJ4Bf/epX3HDDDW7pp5iYGKdLRkYGgYGBbN682Sv95JgasKCggKqqKpqamobUT2eccYbTxWaz0dLSQk1Njdv6KSwmiYfe386UqADCDlaxaVOzR36fCgoK+v198kU/9YlSakAv4JfAnaeUpQNre6n/OfajzxnYr4M6yucA7xnb24A4YzsO2GZsPwUs7PYdZ73eXjk5OcqMlJaWenV//y6tV+Pvekd9vHmv29v2tkt3vvnNbyr7f1v34EsXd+NJl66uLnXTknUq/afvqdp9Rz22H6XM2ydAieol7/R7ai8io0Uk1LENnA9UGEeniEgA8DPgr8b7UUY9RGQ+0KGU2gzsAjJFZKzR9Hxgi7G9FFhkbC8C3upW/l3j7v1ZwGFlXAKwGo6/0N7im7Z4EsJH8ucV1W6f7MPbLt1xrCza1eWe67++dHE3nnR5Z+MePtrcyP/NTyd5zGiP7Qes2SeuXCONAQpFpBxYC7yrlPoAWCgiVcBW7Nc8nzXqRwNlIrIFuAu4FkAptRu4D1gtIhuB6cBvjO88CMwXke3YE+yDRvl72C8DVAN/B24dgutXimGBAdx6zgTW7zzEqqp9vg7HbZx99tkAHnlcVNMzB4628YulldiSwrnRzWNG/QW9+J2XOHr0KCEhA1tNcai0d3RxzsMrGRMazJu3nu221Rl94dIdEWHWrFnO62pDwdcu7sRTLt9/sYwPK/fy7h2zSY/x/HPwZu0TvfidCfDk2uy9MTwogNvPnUh5/SFWbnPfUakvXLqTmZnJZ5995pa2fO3iTjzh8kHFXt7ZuIc7zk3zShIFa/aJTqRewnHX0ttckZNIUuRI/vhxlduulfrKxYE7JzDxtYs7cbfLoePt/OzNCjLjwrh53gS3tt0XVuwTnUj9nGGBAdx+ThobGw6zfEuTr8NxCwsXLgRwDqHReIb7397MoePt/P6qaV+phewGg/7X8RJpaWk+2/dl2QmMixzFo8vdc1TqSxeAYcOGAfDwww8PuS1fu7gTd7os39LIf9bv4pZ5E5gSf4bb2nUFK/aJTqReIjg42Gf7HhYYwB3npVGx6wgfVu4dcnu+dOkew5IlS/qv6EI7/oK7XA4cbeOuf29kUmwo3z93olvaHAhW7BOdSL2Er09DL50ez8ToEH734bYhP4Pvaxdw33VSM7i4C3e4KKW45z+bONLawaPXTHfL0iEDxYp9ohPpV4SgwAB+fEEGNfuO8WpJg6/DGTLf/7594jDHo6sa9/BaaQMfbW7kzgvSmRQb5utwLINOpF4iOto969kMhfmZMeSMj+DRj6s43t4x6HbM4DJ2rP0BuSeeeGJI7ZjBxV0M1aW++Tj3v72ZvJRIbijw3cB7K/aJTqReIj093dchICLcc9EkmlraePazukG3YwYXB0NdWdRMLkNlKC6dXYr/96p90phHrrYRGOCehzcGgxX7RCdSL+GOp3DcQW5yJPMzY/jryi9oPtbe/xd6wCwu3/72tzl27NiQ2jCLizsYisvfP61hbV0z910yhcQI366/ZMU+0Yn0K8iPL8jgWHsHf/6k2tehDAnHDSfHVHKawVGx6zCPfLSNi7JiuTzbktP9+hydSL2EmYZ0pMWEcnVuEs+vqaNm38BnmzeLy9SpUwF49dVXB92GWVzcwWBcjrZ1cPtL64kaHcwDl01123wMQ8GKfaInLfmKsq+ljXMeXsmM5AievX6mr8MZNCLC1KlT2bhxo69DsST/98oG3tywixdvOouzUq23eqc30ZOWmACzJfexocHccd5EVmzbx4qtA3t01EwuZ511Fps2bRr0983kMlQG6vLv0gb+s34Xd5yXZqokasU+0YnUS5hxwbbrzk4hZcxofvXOZto7XB+kbyaXoQ7MN5PLUBmIyxf7jvLztyrIS4nk9nPN9UimFftEJ9KvMMODAvj5xZOp2X+M54rqfB3OoLjkEvv6i2vWrPFxJNbhxMlObn9xPcFBATx2zZk+HerkL+hE6iXy8vJ8HUKPnDsphnkZY3ns4+3sa3FhkS/M5RIQYP8v/Lvf/W5Q3zeTy1Bx1eU3721h854jPHK1jdgzRng4qoFjxT7RidRLNDSY97HMn1+cyYmOTn773pb+K2M+l7Fjx/LGG28M6rtmcxkKrri8uX4XzxXt4MaCFM6dFOOFqAaOFftEJ1Iv4VhK1oxMGBvC/86ZwH/W7+Kz6v391jebi2NBvMFgNpeh0J/Llj1HuPs/G5mZEsldF03yUlQDx4p9ohOpBoDvnzuR8VGj+OkbmzhxstPX4QyIm266CbDmL6C3ONx6klv+VUrYiGH8+X/O1BM1uxn9r+klMjIyfB1Cn4wYFsgDl06l7sBxnljR9xNPZnMJDbWvJfToo48O+LtmcxkKvbl0dSn+36sbaDjYyhPfziY61HzXRbtjxT7RidRLBAZ6f17HgVKQNobLzkzgyVVfUN3U0ms9s7oMZsZ8s7oMht5cnlz1BR9vaeKn35hMbnKkl6MaOFbsE51IvcTmzZt9HYJL/PQbkxk1PIif/KeCrq6en3ozo8stt9wyqO+Z0WWw9OSyclsTj3y0jUts8Vx3drL3gxoEVuwTnUg1X2JMSDA//cZk1tY1s6SoztfhuMydd94JQGtrq48jMQ/VTUe5/cX1ZMSG8eAV5niO3l/RidRLxMbG+joEl7kqJ5FzMsby0Adbe5zUxIwuqan2iYgHuo6TGV0GS3eXQ8fbuXHJOoKHBfCPRbmMGh7kw8gGhhX7RCdSL5GSkuLrEFxGRHjwimkEBwVy52vldJ5yim9mlwcffHBA9c3sMlAcLic7u7j1hTJ2HzrBU9fmkhA+0seRDQwr9olOpF6iqKjI1yEMiJiwEdx3yRTKdh7iH5/WfOkzs7qcf/757NixY0DfMavLYCgqKkIpxS+XVvL5Fwd48Iqp5IyP8HVYA8aKfaITqaZXFkyP54IpMTyyrIqqxt7v4psFxwQm/jY15EBY8nkdLxTv5Oa5E7g8O9HX4XxlcCmRikidiGwSkQ0iUmKU2USkyCh/W0TCjPLhIvKsUV4uIvOM8lDj+47XfhF51PgsWEReEZFqESkWkeRu+77HKN8mIhe42d9rjBxprdMrsJ/iP3DZVEKDg7jjpfXOgfpmdTnnnHMA+Pjjj13+jlldBsOmgwHc985m5mfG8OMLrDcW04El+0Qp1e8LqAPGnFK2DphrbH8P+JWxfRvwrLEdDZQCAT20WQrMMbZvBf5qbF8DvGJsZwLlQDCQAnwBBPYVa05OjtK4l0+2NKrxd72jfv7mJl+H0i+AOu+883wdhtdZV3tApf/0PXXpXwrV8bYOX4fjlwAlqpe8M5RT+wxgtbG9DLjC2M4ElhtJugk4BHxpVmkRSTOS7KdG0QLAcbv1deA8sY/VWAC8rJRqU0rVAtWAJadzLy4u9nUIg+acSdHcNDuF54p28EHFXlO7TJgwgeXLl7tc38wurlLddJQblpQQGQxPL5rByOHWG9DeHSv2iauJVAEfiUipiCw2UlSaQQAAFWRJREFUyiqAS4ztq4AkY7scWCAiQSKSAuR0+8zBQuxHnY6LWQlAPYBSqgM4DER1LzdoMMosh9XHN/7ogklMSzyDH79eTn3z0Fbu9CQDncDE6v3SdOQEi55Zy7DAAH6YE0zk6OG+DmnIWLFPXB1cNksptVtEooFlIrIV++n84yJyL7AUcKzt+wwwGSgBdgCfA6cu83gNcG239z2NFFZ9lH8JI7kvBoiPj2flypWAfWxhaGgo5eX29bqjoqKYMmUKq1fbD6SDgoIoKCigrKyMI0eOAJCbm0tjYyP19fb8nZaWRnBwMBUVFQBER0eTnp7uXDI2ODiY/Px8SkpKnDN75+Xl0dDQ4JxEIyMjg5MnTzrjio2NJSUlxXl3cuTIkeTl5VFcXOz8T5Sfn09tbS179+4FIDMzk87OTrZt2wZAQkICiYmJzr/eISEh5ObmUlRURFubfV7RgoICqqqqaGqyLyWSlZVFW1sb27dvByApKYmYmBjn0g5hYWFkZ2dTWFjoXJlzzpw5VFZWcuDAAb6T2sV9TYonN5wgPHgFQQFCcnIykZGRlJWVARAREYHNZmPVqlUopRAR5s6dS3l5OQcPHgQgOzub5uZm6urq3N5P48aNA+D9999n/vz5/fZTW1ubs18yMjIIDAx0Pllj9n76cMWnPPD5UfYf7+LVm89md8Uap4vNZqOlpYWaGvuIC7P1U1+/T0C/v0++6Kc+6e2cv7cX8EvgzlPK0oG1vdT/HMjs9t4GVJ1S50Mg39gOAvZjT6L3APf0VK+3l1mvkZ44ccLXIbiFtzbsUuPvekf94q0KX4fSK4BavHixS3Wt2i9HT5xUl/6lUE38ybtq1bYmpZR1XU7FrB4M5RqpiIwWkVDHNnA+UGEcnSIiAcDPgL8a70cZ9RCR+UCHUqr7w7MLgZdO2c1SYJGxfSXwiRH4UuAa465+CpAGrO0vZjNSW1vr6xDcwiW2eC7LPIN/fl7HayX1/X/BR/ztb39zqZ4V++XEyU5uXFLCxobD/GlhNnPSxwLWdOkJK3q4co00BigUkXLsSexdpdQHwEIRqQK2AruBZ4360UCZiGwB7uLLp/AAV3N6In0aiBKRauD/gLsBlFKVwKvAZuAD4DallLUmyzRwnFL4AxcnnuTsCVH89M0KNjYc8nU4pzGQBfGs1i/tHV3c8q9S1tQe4JGrbFyY9d/HKa3m0htW9Og3kSqlapRSNuM1RSn1gFH+mFIq3XjdbRxBopSqU0plKKUmK6W+ppTacUp7qUqpraeUnVBKXaWUmqiUmqmUqun22QNKqQlGm++7R1szFAIDhD8tPJOxIcH87/OlLq/15C1+8IMfAHD48GEfR+JeTnZ28YOX17Ni2z5+c9lULj3Tkvdd/RL9ZJOXyMzM9HUIbiMzM5OokGCeujaHg8fbufG5ElrbzXOiEBcXB8BTTz3Vb12r9Et7Rxfff7GM9yv28vOLM1k4c9xpdazi0h9W9NCJ1Et0dpon0QwVh0tWwhk8ds2ZbGw4xP/3yvrTJjfxNa5MYGKFfjlxspOb/1XKh5WN/PKbmdxQ0POkHlZwcQUreuhE6iUcwyz8ge4uF0yJ5effyOTDykYeeNe1VUi9wZVXXukcxtMXZu+X1vZObnquhE+2NvHAZVlcN6v3mZHM7uIqVvTQiVQzZL5XkML1s5J55rPa02aK8hWOG05WPLpx0HLiJN/75zoKq/fzuyun8e288b4OSdMLOpF6iYQE/7kx0JPLz76RyUVZsfz63S28sm6nD6L6Mrm59qeS33zzzT7rmbVfmlpOcM3f1rC2rpk/Xj2dq3NPfTjwdMzqMlCs6KETqZdITPSfKc16cgkMEB69Zjpz08dy93828dYGcyyN/NBDD/X5uRn7pXb/Ma548nNq9h3jH4tyXb47b0aXwfD/t3fmUVVd9x7//LjMyqAgIoJFoxLBhIgYxRFX2sTYJj6bQdNmaCb7MnVl0Jqs12fzkmY1TWPT5jVDjUl8bRrN2IS+RM2kEoITYkAwiooYEUEBqzJ48cJ+f5wjMUTk4r3ee859+7PWXevcffa99/dde/NjT+f3s6MO7Uh9hB0DMXRHd1rCgh385aaxjB/anwffLGF1uX/PA44ZM4bNmzeftY7V2qVk/7+45oVCmp3tLJ83gelpCW5/1mpazhU76tCOVONVwkMcLL1lHBcnx3Dv68V8uO2g32zpbQATf/NB6UHmLFlPnzAH79w1kUtSYv1tksZNtCP1EX379vW3CV6jJy19w4L5n9suJTM5lntfL+btLdU+suzbXHONEdlx69at3daxQrt0dCj+8HEF97xeTEZSDO/eNYmh8X16/T1W0OIN7KhDVIClZcjOzlanouRo/EtLm4t5f91Cwe56HpuVwc05qT63QUSYO3cuy5d3fSrZGrS0uXjozRJWltVy3dhkfjN7NGHB9o4nGqiIyBalVPaZ7ukRqY+wY0Kv7nBXS2RoMEtvyeYH6QNZ9H45T6/eSYePD+1HR0ezYsWKbu/7s112H2pi9nOFrC6v5Vc/HMVT117skRMNlD5mRx3akfoIt2Ia2oTeaAkPcfD8T7OYOy6FP6/ZzX0rvsn95AvOFsCktbXVb+3yj63VXP3nAg43OVl266XcMWUYRlKIcydQ+pgddWhHqjnvhDiC+O2PL+KRKy/kw20Hmbtkg88Cndx9990A1NXVsWPHDl5++WWuv/5WBg0aQWRkZGcAYl/R2tbOwrdLeeCNEkYPjuHDX0zpDIOnsS96jdRHuFwugoPdTUhgbTzRsqqslvvf2EpMRAj/fUMWlw7t72XrDJxOJ8XFxeTnF/Dww78EoE+fVGASzc2TgUHExt7NoUNVhISEnBcburJl3xEWvFVCZX0z904fzv3fH0Gww3tjmUDpY1bVoddILUBFRYW/TfAanmiZMTqRd++aRGRoMDe8tIEX1+3x2rrpnj17WLDgETIzpxIdHceMGfewaNF+4GlgD83Ne2lufg34d+AIEydO6UzncT5xutp5cuUOrnuxEKerg7/fMZ75V6R51YlC4PQxO+rQjtRHnMrHEwh4qiU9KZq8eydxRcZAnly5g58t28zBo54nPCst3cbTTz9Faek9tLUd5NixYtrangUeAoZ9q25ExOdceeWU894uGysb+NGzBby4bg/XZ6ew6v4pTBoef15+K1D6mB11aEeq8QtR4SE895MsHp+Vwea9jVz+TD7vbKnGk6Wm2bP/jfvue5DIyL8A4Wet63B8ztSpU875t3ri0PETPPDGl8xZsoGWtnZevXUcT15zMVHhvllG0PgWvUbqI+rr64mPPz8jEV/jbS1V9c0seLuEzVVHyE0bwK+vyjinA+lgRHu67LKr2bBhCE7n85w5EW0tERGjaGpqoLGx0ataTpxs52/r9/Hsp7twujqYN3UY90wf7pNc84HSx6yqQ6+RWgA7HunoDm9rSY3vw4p5OSz6UTpFVUe44pl8frdqB83Orlm8e8bhcJCXt5zExHyCgp7vplYB2dmTCAoK8poWV3sHKzZ9Te7v1/LEh18xNrUfq+6fwvwr0nziRCFw+pgddWhH6iN8sanhK86HFkeQcNvkoXw2fxpXZSbxwto9TPv9GpZ+Xtnrc6fR0dF8+mkeffs+Dnzynfuhocb6KHiuxelq543NX3P5M/k8/O42BsWGs/zOCSy79VKGDfDto46B0sfsqMN6Zww0/69JiApn8fWZ3DhhCIs/quA3H3zFkvxK5k0dxnXZKcREuLfGeMEFF/D++yuYOXMOra0FGJm8DcLD88nN7W606h6NzW0s3/Q1ywqrOHzcSUZSNC/dnM33RyV4fLBeYz+0I/URKSk9B+a1C77QMmZIP167YzwbKxt45hPDof7h4wp+nDWYGyd8jwsTo3v8jtzcXBYvfpz586+ipWUDEAsc5cSJXYwdOxbonZb2DsXnuw7zVlE1H22v5WS7YsqIeP445xImXhDndwcaKH3Mjjr0ZpOPaGpqsmVUmzPhDy1lB46yrLCKvJIa2lwdpA2M4upLkph50SBS4yLP6sR+/vNf8NprO2lp+QD4mDFjnqK4eA3Qsxanq53CPQ18VF7Hx9vrqG9y0i8yhNljkpkzLoW0xChvSz1nAqWPWVXH2TabtCP1EWvXriU3N9ffZngFf2ppaHLywbaD5H1ZQ9E+I7ldcr8IpowYwIRh/blocAypcX0ICvrGsbpcLqZNm0lRUTrt7ZEsXBjCE0/8F/BdLUdbT7K95hibqxrZtLeRLfuO0HqynT6hDnIvTOCHFw3islEJlozQFCh9zKo6zuZI9dReYyvi+oZxc04qN+ekUn2khTU7DvH5rnr+t6SG5ZuMXFF9Qh2MTIxicGwEg2MjSIgO57bHFrPzJ7P5V0MDYUNe4O0t1bS2udi0q42V9aXUHG2lou44dceMHWMRSBsYxXXZyeSmDWDiBfGEh1jPeWqsgXakPiI6uuc1PbtgFS3J/SK5KSeVm3JSOdnewa66JspqjlJ+4Ci7DzdRduAoH5XX0dbeAUDYrF8S9N5ilu4MJmhvCQBBAvF9D5EYE86k4fGMHBhFWmIUWSn9iIm01+F5q7SLp9hRh57aawKajg7FsRMnaWlrp6XNhdPVQVhwEKEOB+GhQcT1CcMRpHfZNT2jD+RbgIKCAn+b4DXspCUoSIiNDCUpNoLhCVFkJMUwPCGKIXGRJESFs77wC3+b6DXs1C5nw4463HKkIlIlIttE5EsRKTLLMkVkvVn+TxGJNstDReRVs7xERHJP+55QEVkiIhUiskNErjHLw0TkDRHZLSIbRST1tM88YpbvFJErvKjdp7hcvX9Kx6poLdYkULTYUUdv1kinK6XqT3u/FJivlFonIrcBC4D/BO4EUEpdJCIJwEoRGaeU6gD+AziklBopIkHAqWCUtwNHlFLDRWQu8DtgjoikA3OBDCAJ+ERERiqlfBdiXaPRaHrArTVSEakCsk93pCJyDIhRSikRSQFWK6XSReQ5YL1S6jWz3qfAI0qpTSKyH7hQKdXc5ftXA48qpdaLSDBQCwwAHgZQSv22a73ubLXqGmlHRwdBQYGxkqK1WJNA0WJVHd5YI1XARyKyRUTmmWVlwNXm9XXAqccRSoBZIhIsIkOBsUCKiJxK0v24iBSLyFsiMtAsGwzsB1BKuYCjQNzp5SbVZpntKC8v97cJXkNrsSaBosWOOtyd2k9SStWYU/WPRWQHcBvwrIgsAvKANrPuK8AooAjYBxQCLvO3koEvlFIPisiDGKHLb+LMsc7UWcq/henc5wEkJSWxdu1aAIYNG0ZUVBQlJcZRl7i4ODIyMsjPzzfEBwczefJkiouLO3P3ZGdnU1dXx/79hv8eMWIEYWFhlJWVAZCQkMDIkSM7F8TDwsLIycmhqKiIpqYmAMaPH091dTUHDhwAIC0tjdraWhoaGgBITExk6NChndkSIyIiGD9+PBs3bqS11QhwnJOTw969e6mtrQUgPT2d9vZ2du7cCcDgwYNJTk5m48aNgJELPDs7m/Xr13dGz5k8eTIVFRWdgXJHjx6N0+nsDAqRkpLCwIEDOTWCj46OJisri4KCgs51qqlTp1JeXt5pe2ZmJgcPHux8n5qaSv/+/SkuLgagX79+ZGZmsm7dOpRSiAjTpk2jpKSEI0eMA/RZWVk0NjZSVVXl93aqqanp1JKWlobD4WD79u22bKfq6upvtdPx48eprKy0XTs5nc4e/5780U5nRSnVqxfwKMba6OllI4FN3dQvBNIxnGIzEGSWpwDl5vVqIMe8DgbqzfqPYCwL0LVed6+xY8cqK7JmzRp/m+A1tBZrEiharKoDKFLd+J0ep/Yi0kdEok5dA5cDZeboFHPT6FfAi+b7SLMeIvIDwKWU2m4a8k8g1/zqy4Dt5nUecIt5fS3wmVk/D5hr7uoPxQjhs6nH/w4WJDMz098meA2txZoEihY76nBnjXQgUCAiJRhO7AOl1CrgBhGpAHYANcCrZv0EoFhEvgIWYkzdT7EQeFRESs3yh8zyl4E4EdkNPMg3m0zlwJsYDncVcI+y6Y798ePH/W2C19BarEmgaLGjjh7XSJVSlcB3/kUopf4E/OkM5VVAWjfftQ+YeobyExgbVmf6zBPAEz3ZaXUqKysZMmSIv83wClqLNQkULXbUYb0zBhqNRmMzAu5ZexE5jHFawGrEY2yiBQJaizUJFC1W1fE9pdSAM90IOEdqVUSkSHVzmNduaC3WJFC02FGHntprNBqNh2hHqtFoNB6iHanvWOJvA7yI1mJNAkWL7XToNVKNRqPxED0i1Wg0Gg/RjtTLiMgMMwj1bhF5uJs614vIdhEpF5HXfW2ju/SkRUSGiMgaEdkqIqUiMtMfdvaEiLwiIodEpKyb+yIiz5o6S0Uky9c2uosbWn5qaigVkUIRsezzlj1pOa3eOBFpF5FrfWVbr+nuIXz96v0LcAB7gGFAKEZIwfQudUYAW4F+5vsEf9vtgZYlwF3mdTpQ5W+7u9EyFcgCyrq5PxNYiREoZwKw0d82e6Bl4ml960o7azHrOIDPgA+Ba/1tc3cvPSL1LpcCu5VSlUqpNmAFMKtLnTuB55RSRwCUUod8bKO7uKNFAadSPsZgxFywHEqpfKDxLFVmAX9VBhuAWBEZ5BvrekdPWpRShaf6FrABI3SlJXGjXQDuA94BrPp3AuipvbdxJxD1SGCkiHwhIhtEZIbPrOsd7mh5FLhRRKoxRgz3+cY0rxMwAcS7cDvGSNuWiMhgYDZmZDkrox2pd3EnEHUwxvQ+F7gBWHpa9gAr4Y6WG4BlSqlkjOnx38ywinbDrQDidkJEpmM40oX+tsUD/ggsVDaI+Nab5Heanqnmm5QrYEyruk53q4ENSqmTwF4R2YnhWDf7xkS3cUfL7cAMAGXk2wrHeE7a0tOwM+COVtsgIhdjJKe8UinV4G97PCAbWCEiYPSrmSLiUkq951+zvosdRw9WZjMwQkSGikgoRgbUvC513gOmA4hIPMZUv9KnVrqHO1q+xgjQjYiMAsKBwz610jvkATebu/cTgKNKqYP+NupcEJEhwLvATUqpCn/b4wlKqaFKqVSlVCrwNnC3FZ0o6BGpV1FKuUTkXoyUKA7gFaVUuYg8hpGmIM+8d7mIbAfagQVWHDW4qeUh4CUReQBjKvwzZW61WgkRWY6xlBJvruf+GggBUEq9iLG+OxPYDbQAt/rH0p5xQ8sijMSRz5sjOZeyaAAQN7TYBv1kk0aj0XiIntprNBqNh2hHqtFoNB6iHalGo9F4iHakGo1G4yHakWo0Go2HaEeq0Wg0HqIdqUaj0XiIdqQajUbjIf8HyqrGDJAJUd0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 360x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "# 画图\n",
    "errors = rc.cv_values_.mean(axis=0)\n",
    "plt.figure(figsize=(5,5))\n",
    "plt.grid(ls='--')\n",
    "plt.plot(rc_alphas, errors)\n",
    "\n",
    "# 标注最小alpha的位置\n",
    "min_error_idx = errors.argmin()\n",
    "optimal_alpha = rc_alphas[min_error_idx]\n",
    "min_error = errors[min_error_idx]\n",
    "# plt.text(optimal_alpha, min_error, f'alpha: {rc_alphas[min_error_idx]:.3f}')\n",
    "plt.annotate(f'optimal alpha: {optimal_alpha:.3f}', \n",
    "             xy=(optimal_alpha, min_error), \n",
    "             xytext=(optimal_alpha-0.1, min_error+300),\n",
    "             arrowprops=dict(facecolor='blue', shrink=0, width=0))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1200974.85666336, 1201039.24893512, 1201103.64413847])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rc.cv_values_[0,0:3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型性能评估\n",
    "用RMSE对模型进行性能评估"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "最小二乘线性回归对应的RMSE为857.293\n",
      "岭回归对应的RMSE为855.053\n",
      "LASSO对应的RMSE为859.009\n"
     ]
    }
   ],
   "source": [
    "# 计算各个模型的RMSE\n",
    "rmse_lr = np.sqrt(mean_squared_error(y_test, y_lr_test_pred))\n",
    "rmse_rc = np.sqrt(mean_squared_error(y_test, y_rc_test_pred))\n",
    "rmse_ls = np.sqrt(mean_squared_error(y_test, y_ls_test_pred))\n",
    "\n",
    "print(f'最小二乘线性回归对应的RMSE为{rmse_lr:.3f}')\n",
    "print(f'岭回归对应的RMSE为{rmse_rc:.3f}')\n",
    "print(f'LASSO对应的RMSE为{rmse_ls:.3f}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对比三种模型得到的各个特征的系数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>column</th>\n",
       "      <th>lr_coef</th>\n",
       "      <th>rc_coef</th>\n",
       "      <th>ls_coef</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>yr</td>\n",
       "      <td>2.053785e+03</td>\n",
       "      <td>2058.645163</td>\n",
       "      <td>2056.973815</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>season_1</td>\n",
       "      <td>-4.542393e+16</td>\n",
       "      <td>-958.614189</td>\n",
       "      <td>-1169.754693</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>season_2</td>\n",
       "      <td>-4.542393e+16</td>\n",
       "      <td>138.323349</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>season_3</td>\n",
       "      <td>-4.542393e+16</td>\n",
       "      <td>86.315408</td>\n",
       "      <td>-54.961105</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>season_4</td>\n",
       "      <td>-4.542393e+16</td>\n",
       "      <td>733.975432</td>\n",
       "      <td>542.294062</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>mnth_1</td>\n",
       "      <td>-1.823947e+16</td>\n",
       "      <td>-319.821241</td>\n",
       "      <td>-166.992083</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>mnth_2</td>\n",
       "      <td>-1.823947e+16</td>\n",
       "      <td>-137.420627</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>mnth_3</td>\n",
       "      <td>-1.823947e+16</td>\n",
       "      <td>183.663785</td>\n",
       "      <td>226.256787</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>mnth_4</td>\n",
       "      <td>-1.823947e+16</td>\n",
       "      <td>17.661463</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>mnth_5</td>\n",
       "      <td>-1.823947e+16</td>\n",
       "      <td>289.506526</td>\n",
       "      <td>212.331202</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>mnth_6</td>\n",
       "      <td>-1.823947e+16</td>\n",
       "      <td>178.245855</td>\n",
       "      <td>46.166155</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>mnth_7</td>\n",
       "      <td>-1.823947e+16</td>\n",
       "      <td>-336.825142</td>\n",
       "      <td>-466.190690</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>mnth_8</td>\n",
       "      <td>-1.823947e+16</td>\n",
       "      <td>125.718031</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>mnth_9</td>\n",
       "      <td>-1.823947e+16</td>\n",
       "      <td>697.464905</td>\n",
       "      <td>635.025890</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>mnth_10</td>\n",
       "      <td>-1.823947e+16</td>\n",
       "      <td>149.385841</td>\n",
       "      <td>167.251377</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>mnth_11</td>\n",
       "      <td>-1.823947e+16</td>\n",
       "      <td>-382.071224</td>\n",
       "      <td>-291.418262</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>mnth_12</td>\n",
       "      <td>-1.823947e+16</td>\n",
       "      <td>-465.508171</td>\n",
       "      <td>-348.224929</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>weathersit_1</td>\n",
       "      <td>-4.049715e+16</td>\n",
       "      <td>860.056963</td>\n",
       "      <td>518.176964</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>weathersit_2</td>\n",
       "      <td>-4.049715e+16</td>\n",
       "      <td>313.180766</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>weathersit_3</td>\n",
       "      <td>-4.049715e+16</td>\n",
       "      <td>-1173.237729</td>\n",
       "      <td>-1497.712180</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>weekday_0</td>\n",
       "      <td>9.651054e+15</td>\n",
       "      <td>-99.444479</td>\n",
       "      <td>-283.955791</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>weekday_1</td>\n",
       "      <td>3.766759e+16</td>\n",
       "      <td>-170.650754</td>\n",
       "      <td>-138.509538</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>weekday_2</td>\n",
       "      <td>3.766759e+16</td>\n",
       "      <td>-87.185003</td>\n",
       "      <td>-56.300294</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>weekday_3</td>\n",
       "      <td>3.766759e+16</td>\n",
       "      <td>-12.447451</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>weekday_4</td>\n",
       "      <td>3.766759e+16</td>\n",
       "      <td>-11.309801</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>weekday_5</td>\n",
       "      <td>3.766759e+16</td>\n",
       "      <td>16.224018</td>\n",
       "      <td>23.627144</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>weekday_6</td>\n",
       "      <td>9.651054e+15</td>\n",
       "      <td>364.813470</td>\n",
       "      <td>164.554837</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>temp</td>\n",
       "      <td>2.862395e+03</td>\n",
       "      <td>1805.193903</td>\n",
       "      <td>2710.168009</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>atemp</td>\n",
       "      <td>7.862455e+02</td>\n",
       "      <td>1534.930045</td>\n",
       "      <td>932.436858</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>hum</td>\n",
       "      <td>-1.552559e+03</td>\n",
       "      <td>-1307.067839</td>\n",
       "      <td>-1438.506903</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>windspeed</td>\n",
       "      <td>-1.351903e+03</td>\n",
       "      <td>-1213.545701</td>\n",
       "      <td>-1277.867098</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>holiday</td>\n",
       "      <td>-2.801653e+16</td>\n",
       "      <td>-516.389925</td>\n",
       "      <td>-733.192926</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>workingday</td>\n",
       "      <td>-2.801653e+16</td>\n",
       "      <td>251.020934</td>\n",
       "      <td>32.293462</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          column       lr_coef      rc_coef      ls_coef\n",
       "0             yr  2.053785e+03  2058.645163  2056.973815\n",
       "1       season_1 -4.542393e+16  -958.614189 -1169.754693\n",
       "2       season_2 -4.542393e+16   138.323349     0.000000\n",
       "3       season_3 -4.542393e+16    86.315408   -54.961105\n",
       "4       season_4 -4.542393e+16   733.975432   542.294062\n",
       "5         mnth_1 -1.823947e+16  -319.821241  -166.992083\n",
       "6         mnth_2 -1.823947e+16  -137.420627    -0.000000\n",
       "7         mnth_3 -1.823947e+16   183.663785   226.256787\n",
       "8         mnth_4 -1.823947e+16    17.661463    -0.000000\n",
       "9         mnth_5 -1.823947e+16   289.506526   212.331202\n",
       "10        mnth_6 -1.823947e+16   178.245855    46.166155\n",
       "11        mnth_7 -1.823947e+16  -336.825142  -466.190690\n",
       "12        mnth_8 -1.823947e+16   125.718031     0.000000\n",
       "13        mnth_9 -1.823947e+16   697.464905   635.025890\n",
       "14       mnth_10 -1.823947e+16   149.385841   167.251377\n",
       "15       mnth_11 -1.823947e+16  -382.071224  -291.418262\n",
       "16       mnth_12 -1.823947e+16  -465.508171  -348.224929\n",
       "17  weathersit_1 -4.049715e+16   860.056963   518.176964\n",
       "18  weathersit_2 -4.049715e+16   313.180766    -0.000000\n",
       "19  weathersit_3 -4.049715e+16 -1173.237729 -1497.712180\n",
       "20     weekday_0  9.651054e+15   -99.444479  -283.955791\n",
       "21     weekday_1  3.766759e+16  -170.650754  -138.509538\n",
       "22     weekday_2  3.766759e+16   -87.185003   -56.300294\n",
       "23     weekday_3  3.766759e+16   -12.447451     0.000000\n",
       "24     weekday_4  3.766759e+16   -11.309801     0.000000\n",
       "25     weekday_5  3.766759e+16    16.224018    23.627144\n",
       "26     weekday_6  9.651054e+15   364.813470   164.554837\n",
       "27          temp  2.862395e+03  1805.193903  2710.168009\n",
       "28         atemp  7.862455e+02  1534.930045   932.436858\n",
       "29           hum -1.552559e+03 -1307.067839 -1438.506903\n",
       "30     windspeed -1.351903e+03 -1213.545701 -1277.867098\n",
       "31       holiday -2.801653e+16  -516.389925  -733.192926\n",
       "32    workingday -2.801653e+16   251.020934    32.293462"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "features_names = X.columns\n",
    "df_coef = pd.DataFrame(dict(column=features_names, lr_coef=lr.coef_, rc_coef=rc.coef_, ls_coef=ls.coef_))\n",
    "df_coef"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "总结：从上面的结果可以看出岭回归算法在测试集上的表现最好，岭回归和lasso回归都对参数有正则，让参数的浮动范围变小，并且lasso还会起到筛选特征的作用，上面的特征系数很多都是0，但是负面的作用就是利用的特征少了，更适用于特征维数过多，需要过滤掉某些相关性弱的特征的情况。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
